V An experimental comparison of using cause - effect diagrams and simple memos in software project retrospectives

Root cause analysis (RCA) is a recommended practice in retrospectives and cause-effect diagram (CED) is a commonly recommended technique for RCA. Our objective is to evaluate whether CED improves the outcome of RCA and the perceptions of retrospective participants. We conducted a controlled experiment with eleven student software project teams by using two-by-two crossover design resulting in total of 22 experimental units. Two visualization techniques of underlying causes were compared: CED and a simple memo, i.e. a structural list of causes. We used the output of RCA, questionnaires, and group interviews in order to compare the two techniques. CED increased the total number of causes with medium effect size. CED also increased the links between causes, thus, suggesting more structured analysis of problems. Furthermore, the participants perceived that CED improved organizing and outlining the detected causes. The implication of our results is that using CED in the RCA of retrospectives is recommended, yet, not mandatory as the groups also performed quite well with the structural list. CED is visually more attractive and more effective than the structural list, but it is somewhat harder to read and requires specific software tools increasing the burden of adaptability.